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Abstract 

The purpose of this study was to establish the psychometric properties of the Attitudes 
Toward Research scale (ATR), including the validity> of the scale through confirmatory factor 
analysis. After slightly altering the scale by deleting 19 items, and by narrowing down the 
subscales to four, the fit of the model was established. The revised version of the A TR scale 
(R-ATR) includes 13 items, which measure Research usefulness, Research anxiety, and 
Positive predisposition towards research. Overall, the analyses performed on the R-A TR 
suggest that it has strong psychometric properties, and that it can prove helpful to 
researchers interested in examining issues related to research methods attitudes, as well as to 
practitioners to be used for evidence-based practice. 


Given that evidence-based practices are a cornerstone of the accountability movement, 
professionals need to demonstrate both their ability to access research-based knowledge as 
well as their ability to apply that knowledge in real world situations. Consequently, during the 
last decade, special attention has been paid to exposing undergraduate students to research 
experiences and research courses (Lopatto, 2007; Seymore, Huner, Laursen, & Deantoni, 

2004) . In the field of education, drawing teachers into the research process appears to provide 
a vital foundation for the development of teaching as an evidence-based profession 
(Karagiorgi & Papanastasiou, 2012). Teacher research has been viewed as a tool enabling the 
transformation of educational practice, while making important contributions to the 
knowledge base in education. Practitioner research in particular, appears to hold the potential 
to contribute to school improvement (Briggs & Coleman, 2007). 

Elowever, combining the role of the teacher as well as the researcher is not an easy endeavor 
(Ruthven, 2005). According to West (2011), the divide between research and practice has 
emerged since the 1930s when University professors began distinguishing themselves as 
teacher educators or as researchers. Even today, teachers are criticized as being research 
averse (Fusarelli, 2008) while the role of the teacher as a research-based professional still 
resembles an ideal (Westbury, Hansen, Kansanen & Bjorkvist, 2005). 

In an attempt to strengthen the link between teachers and research, many Universities, 
especially in Europe, require from undergraduate students in the field of education to enroll 
in research methods courses. However, despite the great importance of such courses, research 
methods always tend to be one of the least favorite courses for most students who often 
perceive them as an obstacle in their studies (Onwuegbuzie, Leech, Murtonen & Tahtinen, 

2005) . Others dislike research because they do not see themselves as researchers, while 
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others do not feel empowered to understand and use research (Lodico, Spaulding & Voegtle, 
2004). Students also frequently complain that research methods are very demanding, 
overwhelming and difficult courses (Lodico, Spaulding & Voegtle, 2004; Onwuegbuzie, 

Slate, Patterson, Watson & Schwartz, 1998). What makes matters worse is that many students 
tend to confuse research with statistics, which leads many of them to conclude that poor 
quantitative skills will prevent them from doing well in the course, although quantitative 
skills are relevant to only part of a research course. Therefore, students in research methods 
courses carry with them many of the fears and anxieties that are associated with statistics. In 
turn, these attitudes that are related to statistics may interfere with the student’s learning in 
research and may prevent them from being able to understand, or become involved in 
research in their daily lives or in their future professional careers. 

Although numerous studies have examined issues related to statistics attitudes, anxiety and 
achievement (Nasser, 2004; Onwuegbuzie, 2003; Onwuegbuzie & Wilson, 2003; Pan & 

Tang, 2004), very little research has specifically examined where students stand in relation to 
research methods courses. Some of the few studies that have grappled with research attitudes 
have dealt with quantitative methods courses (Murtonen, 2005; Murtonen & Lehtinen, 2003). 
In these cases however, it is hard to disassociate the quantitative component of the course 
with the attitudes of students toward research per se. 

A relatively new trend in the research literature has focused on the effects of research 
experiences on undergraduate students (Korkmaz, Cole & Buckley, 2010; Russell, Hancock, 
McCullogh, 2007; Seymour, Hunter, Laursen & Deantoni, 2004). However, the results of 
such studies are difficult to generalize to the wider undergraduate student population due to 
the fact that such students a) are enrolled in STEM majors (Technology, Engineering and 
Mathematics) instead of education, and b) in most cases voluntarily agree to participate in 
such programs. Therefore, the experiences of such students vary significantly from students 
who are obliged to enroll in research methods classes as part of their undergraduate degree 
program in teacher education. The current study will move beyond these limitations by 
examining the psychometric properties of an instrument measuring teacher-education 
undergraduate student’s attitudes towards research. 

One of the few psychometric measures that currently exist that measure undergraduate 
students’ attitudes towards research is the Attitudes Toward Research scale (ATR) 
(Papanastasiou, 2005). Since the Attitudes Toward Research scale is a new measure, very 
little information is known about its psychometric properties. Therefore, the purpose of this 
study is to validate the scores of the ATR scale through confirmatory factor analysis. The 
factor structure of a scale is an essential aspect in establishing its construct validity. The 
validation of the scores of such a scale can prove helpful to researchers who might be 
interested in using this scale to examine issues related to research methods attitudes more 
carefully, as well as to professors of educational research in an effort to understand students’ 
difficulties and find ways to overcome the multiplicity of reasons behind these difficulties. 
The importance of such studies lies in increasing the impact of research methods courses so 
as to get a step closer to the ultimate goal of training research-oriented teachers. This is 
especially significant nowadays where teachers are urged to use evidence-based practices. 
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Attitudes towards research scale (A TR) 

Attitudes in general, according to the tripartite framework originally presented by McGuire 
(1969), may be defined as the cognitive, affective and behavioral predispositions toward a 
concept, a situation, an object, a course, etc, although these three components cannot always 
be separated. Based on this framework, the Attitudes Toward Research Scale is a self report 
measure of students’ attitudes towards the field of research, regardless of their research 
orientation (quantitative, qualitative or mixed methods). This measure that exists in both 
Greek and English, consists of 32 Likert scale items whose scales range from 1 to 7. The 
value of 1 stands for strongly disagree, while 7 stands for strongly agree. An exploratory 
factor analysis of the data in a previous sample has identified the existence of five factors, 
those of “Research usefulness” (a=.919), “Research anxiety” (a=.918), “Positive research 
predisposition” (a=.929), “Relevance to life” (a=.767), and “Research difficulty” (a=.711) 
(Papanastasiou, 2005). The factor of Research usefulness measures the student’s perceptions 
in reference to how useful they perceived that research would be in their professional lives. 
The Research anxiety factor measures the negative feelings of stress and anxiety felt by the 
students in relation to research. The third factor of Positive research predispositions, 
measures the existence of positive feelings and interest towards research. The factor of 
Relevance to life measures the student’s perceptions of whether research can be applied in the 
student’s daily lives. Finally, the factor of Research difficulty identifies the problems that 
students face with various aspects of research. 

A more recent study examining the psychometric properties of the ATR scale with the use of 
Rasch analysis has identified two items that do not have adequate fit compared to the rest of 
the items of the ATR scale (Papanastasiou & Schumacker, 2010). More specifically, Item 6 
stating, “I feel insecure concerning the analysis of research data”, and item 11 stating “I have 
trouble with arithmetic” did not have adequate fit. By taking a closer look at items 6 and 11 it 
is clear that both items are related to the statistical analysis of data rather than attitudes 
toward research. Consequently, a decision was made to drop these items from the current 
scale since the instrument pertains to research methods courses as a whole regardless of their 
focus (quantitative, qualitative or mixed) l 1 . Since two items had to be dropped from the 
scale though, the construct validation as well as the reliability of the scores from the scale had 
to be re-examined. The results of a confirmatory factor analysis could strengthen the 
construct validity of the ATR scale and encourage more researchers, research instructors and 
practitioners to utilize this scale. 


Methods 

The sample of this study consisted of 317 undergraduate students who had been taking an 
educational research methods course at a European University in the Republic of Cyprus. The 
specific course that the students were enrolled in was a three credit course titled 
‘Methodology of Educational Research’ and was a compulsory course for all students 
studying elementary education, or kindergarten education at this University. All students in 


1 Efforts are underway to create ATR subscales that will focus on the orientation of research methods courses. 
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the above majors were required to enroll in this research methodology course, which typically 
occurs during the sophomore year of their studies. This course was offered by the same 
instructor both, in the spring and fall semesters of every academic year. The sample of this 
study included the students who had enrolled in this course in a period of four semesters. 

The questionnaire was administered on a volunteer basis to all students enrolled in the course. 
The data were collected towards the end of each semester, and all students that had attended 
the class on each day that the questionnaire was administered had participated in the study. 

Of the sample, 89.8% were female which is representative of the gender breakdown of the 
students in the college of education at this University. The main reason for the discrepancy in 
the gender breakdown is due to the large majority of female students who follow the majors 
of elementary and kindergarten education. The majority of the students were sophomores 
(69.84%), 20.66% were juniors, 6.98% were seniors, and another 1.97% were in their 5+ year 
of their studies. For the purpose of this study the data were analyzed with the use of the 
structural equation modeling software AMOS 18 (Arbuckle, 2009), as well as with the use of 
SPSS. 

The hypothesized model that was originally analyzed in AMOS, included the 30 items from 
the ATR scale that were treated as observed variables, and are represented in the 
measurement model with rectangles. Five latent variables were also created to represent the 
five factors of the scale. All latent variables are represented with ovals. The arrows pointing 
to each rectangle/observed variable represent the errors in the measurement of each of those 
observed variables. These error terms are very important to be included in the model since no 
assumptions are made that the variables used in this study are perfectly reliable. Finally, the 
two-headed arrows in the model represent the correlations between the five factors. 

The estimation method that was used for the analysis of the model was the maximum 
likelihood estimation procedure. The maximum likelihood estimation was preferred to that of 
the generalized least squares estimation since it leads to less biased parameter estimates and 
more accurate fit indices (Olsson, Foss, Troye, & Howell, 2000). Both absolute and 
incremental fit indices were used in this study, in addition to a parsimony adjusted index. The 
absolute fit indices that were used, that assess how well the model reproduces the sample 
matrix, were the chi-square (% 2 ), and the chi square divided by the degrees of freedom ratio 
(X 2 /df). The incremental fit indices that were used, that assess model fit relative to a baseline 
model, were the Normed Fit Index (NFI), the Relative Fit Index (RFI), the Tucker-Lewis 
Index (TLI), the Comparative Fit Index (CFI), the Incremental Fit Index (IFI). Finally, a 
parsimony adjusted index, the Akaike Information Criterion (AIC), was used to be able to 
compare models with differing numbers of latent variables. 

Results 

Psychometrically, the score distributions statistics on all items should be checked to ensure 
that they are normally distributed, with no floor or ceiling effects. Table 1 presents the means 
and standard deviations of the 30 items included in the ATR scale. Some items were recoded 
however, so that a higher response on each question corresponded to more positive attitudes 
toward research. All of the items had an average score between 3.31 and 5.22 on a 7 point 
scale. The standard deviations of all items were reasonable and ranged from 1.33 to 1.86, 
indicating that no item had a strong floor or ceiling effect. Overall, all items were 
approximately normally distributed with their skewness ranging from 0.02 to -0.67, and their 
kurtosis ranging from 0.06 to -0.95. 
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Confirmatory factor analysis 

The hypothesized 30-item, five-factor model was originally examined to test whether the 
model fit the data. The results of this analysis did not support the fit of the model since only 
the x 2 /df ratio was in the acceptable range (x 2 /df=2.98) based on Hu and Bentler’s cut-off 
criteria (1999). Therefore, the scale had to be re-examined in order to improve the quality of 
the factors. In an attempt to remove the items with large amounts of error variance, a decision 
was made to delete any items with standardized coefficients that fell below 0.70 due to their 
low levels of reliability. This allowed for the error variances for the remaining items to be 
below 0.50 for each observed variable. Based on this criterion, 17 items were deleted from 
the scale. A consequence from this process is that two factors were deleted, those of 
Relevance to life and Research difficulty. The resulting model included three latent variables; 
those of ‘Research usefulness’ which included 4 items, ‘Research anxiety’ which included 5 
items, and ‘Positive research predispositions’ which included 4 items. The latent variables 
were also allowed to correlate with each other since no assumptions are made that these three 
factors are not related to each other. The revised version of the ATR scale is presented in 
Figure 1. 

Once the revised model was analyzed, the fit indices had to be examined to determine 
whether the data fit this revised measurement model. The results that are presented in Table 2 
were very encouraging since most of the fit indices supported the fit of the model. With the 
exception of the % 2 (that is influenced by sample size), the other fit indices supported the fit of 
the model. The TLI (TLI=0.95), the IFI (IFI=0.97), the NFI (NFI=0.95) as well as the CFI 
(CFI=0.97) all had values equal or higher than 0.95 which strongly support the fit of the 
model. The RFI with a value of 0.92 was slightly below the 0.95 cutoff, but still supported 
the fit of the model since this value ranged between 0.90 and 0.95 (Hu & Bentler, 1999). In 
addition, the AIC had a lower value in the revised model, thus supporting the revised ATR 
scale (R-ATR) as the more parsimonious model. A chi-square difference test (A^ 2 ) was 
calculated between the original and revised factor models to determine whether the revised 
model was significantly better than the original one. Overall, the fit of the revised model was 
significantly better than the original model (A f = 1015.60, Adf =333, p<0.0001). 

Once the fit of the model was established, it was possible to pay closer attention and interpret 
the path coefficients of the final model. As presented in Table 3, all unstandardized 
coefficients were statistically significant at the 0.001 level. Moreover, no standardized 
regression weights were lower than 0.70 which further strengthened the validity of the scale’s 
results. In terms of the factor of Research usefulness, this factor had the strongest effect on 
items q20 “The skills I have acquired in research will be helpful to me in the future” since the 
factor explained 69% of its variance. The factor Research usefulness had the second largest 
effect on item q8 “Research is useful for my career” since it explained 62% of its variance. 
The factor Research anxiety had the strongest effects on item ql 6 “Research courses are 
stressful” explaining 79% of its variance and on item ql 8 “Research courses make me 
nervous” explaining 77% of its variance. Finally, the Positive research predisposition factor 
had the strongest effect on item q3 “I enjoy my research course(s)” (79%) and on item ql2 “I 
love research courses” (75%). 
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Correlations between the ATR factors 

Table 3 presents the correlations between the three factors of the revised model. The highest 
correlation was between the factors of “Research anxiety” and “Positive research 
predisposition” (r=0.62). The smallest correlation was between the factors of “Research 
usefulness” and “Research difficulty” (r=0.18). The correlation between “Positive research 
predispositions” and “Research usefulness” was equal to 0.54. The low to moderate 
correlations between the three factors provide discriminant validity evidence among the three 
factors. 

Reliability Results 

Once the factor structure of the revised version of the ATR scale was established, the 
estimates of internal consistency were also calculated by using Cronbach’s coefficient alpha. 
The results of these analyses show that the reliability of the three factors was slightly lower 
than the results reported for the original 32-item version of the scale. However, the latest 
reliability coefficients of the R-ATR were still in the very good to excellent range. More 
specifically, the reliability of the Research usefulness scale was in the excellent range 
(a=0.90) even though it dropped from 0.92 to 0.90 (Table 5). The reliability of the Positive 
research predisposition factor was also in the excellent range (a=0.92). The Research anxiety 
factor had the largest decrease in size (a=0.86) although its reliability was still in the very 
good range. 


Discussion and Implications 

The Attitudes Toward Research scale, which currently exists in Greek and in English, is one 
of the few instruments created with a focus on measuring student attitudes towards the 
subject area of research methods. Since the creation of the scale however, its results have not 
been cross validated with samples of undergraduate students in the field of education. 
Therefore, the purpose of this study was to establish the psychometric properties of the ATR 
scale, including the validity of the scores of the scale through confirmatory factor analysis. 
The original version of the questionnaire included 32 items that were broken down into five 
factors. The confirmatory factor analysis that was performed on the original version of the 
questionnaire showed that the fit of the model was not adequate. After deleting 19 items in 
total due to their amounts of errors variance, and by narrowing down the factors to three 
distinct factors, the fit of the model was established. The revised version of the questionnaire 
includes 13 items, which measure Research usefulness, Research anxiety, and Positive 
research predisposition. These items were all normally distributed and did not show any 
strong floor or ceiling effects. 

The reliability of the scores from the revised version of the ATR scale (R-ATR) was high, 
although it dropped slightly compared to the original full version of the ATR scale. This is a 
very small setback when taking into consideration that 19 items have been eliminated from 
the revised version of the scale, especially since this allowed for the parsimony of the model 
to improve. 

At this point it is important to acknowledge some of the limitation of the study. The main 
limitation lies in the fact that a larger proportion of females than males have participated in 
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this study, mainly due to the fact that the teaching profession tends to be dominated by 
female teachers. Once more data are collected from male students in the field of education, 
future research could involve the examination of group differences (invariance tests) to 
determine whether attitudes toward research are related to variables such as gender or age. 

Despite the limitation mentioned above however, the overall analyses performed on the R- 
ATR suggest that it has strong psychometric properties. Therefore, a conclusion can be 
reached that the R-ATR is a promising self-report measure that can be used to assess college 
student’s attitudes towards the field of research, and more specifically in assessing positive 
predispositions towards the field of research, research anxiety and research usefulness within 
the context in which the data were obtained from. This scale can therefore be utilized within 
this context by researchers examining the concepts of research methods attitudes, or by 
professors of educational research who might want to look deeper into the attitudes of their 
students on this issue. Further analyses of the scale, to determine its degree of construct 
validity within the USA are also underway in an effort to broaden its use, for research and 
teaching purposes. 

In conclusion, through a systematic research agenda, future studies could try to establish links 
between research attitudes and teacher involvement in research in the workplace in an 
attempt to strengthen the development of teaching as an evidence-based profession. Research 
attitudes could be examined as a predictor of further research activities in an attempt to make 
further progress in the establishment of the educational field as an evidence-based profession. 
This is especially important nowadays due to the push that exists towards accountability, 
school effectiveness and school improvement. 
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Table 1. 

Descriptive statistics of the A TR scale 


Factor 

Items 

Mean 

SD 

Skewness 

Kurtosis 

Research usefulness 






q8 Research is useful for my career. 

5.13 

1.39 

-0.67 

0.06 


ql4 Research is connected to my field of study. 

4.76 

1.61 

-0.51 

-0.51 


q20 The skills I have acquired in research will 
be helpful to me in the future 

5.22 

1.33 

-0.67 

0.24 


q24 Research should be indispensable in my 
professional training 

4.77 

1.48 

-0.30 

-0.51 

Anxiety 

ql Research courses make me anxious. * 

3.31 

1.79 

0.36 

-0.86 


q7 Research courses scare me. * 

4.01 

1.81 

-0.10 

-1.02 


ql6 Research courses are stressful. * 

3.64 

1.86 

0.15 

-1.12 


ql8 Research courses make me nervous. * 

4.02 

1.84 

-0.09 

-1.08 


q28 Research courses are difficult. * 

3.65 

1.67 

0.02 

-0.95 

Positive research 





predispositions 

q3 I enjoy my research course(s). 

3.53 

1.56 

0.11 

-0.73 


ql2 I love research courses. 

3.38 

1.60 

0.28 

-0.63 


ql3 1 find research courses interesting. 

4.23 

1.46 

-0.23 

-0.48 


q30 Research courses are pleasant. 

3.62 

1.49 

0.12 

-0.52 


* Recoded items 
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Table 2. 

Fit indices of the structural models 



Flypothesized model 

Revised model (R-ATR) 

Model difference 

x 2 

1175.83 (df=395, p<0.000) 

160.23 (df=62, p<0.000) 

Af= 1015.60 

X 2 /df ratio 

2.98 

2.58 

Adf =333, p<0.0001 

NFI 

0.82 

0.95 


RFI 

0.79 

0.92 


CFI 

0.87 

0.97 


IFI 

0.88 

0.97 


TLI 

0.85 

0.95 


AIC 

1375.83 

244.23 
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Table 3. 

Path coefficients of final Revised-ATR scale 


Factor 

Path 

Items/Factors 







Standardized 

Unstandardized 

S.E. 




Estimates 

Estimates 


Research usefulness 


Pos. predisposition 

0.54 

0.76** 

0.11 

Research anxiety 


Pos. predisposition 

0.62 

0.99** 

0.13 



Research 




Research anxiety 


usefulness 

0.18 

0.24** 

0.09 

Research usefulness 


q20 

0.83 

1.01** 

0.07 

Research usefulness 


q8 

0.79 

1.00 


Research usefulness 


ql4 

0.73 

1.07** 

0.08 

Research usefulness 


q24 

0.74 

0.99** 

0.08 

Research anxiety 


q 18 

0.88 

1.29** 

0.08 

Research anxiety 


ql6 

0.89 

1.32** 

0.08 

Research anxiety 


q7 

0.83 

1.20** 

0.08 

Research anxiety 


qi 

0.86 

1.23** 

0.08 

Research anxiety 


q28 

0.75 

1.00 


Pos. predisposition 


ql2 

0.87 

1.09** 

0.06 

Pos. predisposition 


q30 

0.85 

1.00 


Pos. predisposition 


q 13 

0.82 

0.94** 

0.05 

Pos. predisposition 


q3 

0.89 

1.09** 

0.05 


** p<0.001 
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Table 4. 

ATR item correlations 



q3 

ql2 

q 13 

q30 

qi 

q7 

ql6 

q 18 

q28 

q8 

ql4 

q20 

q3 













ql2 

.77 












ql3 

.73 

.72 











q30 

.76 

.72 

.71 










qi 

.48 

.40 

.39 

.39 









q7 

.46 

.45 

.44 

.45 

.69 








q 16 

.49 

.40 

.39 

.39 

.78 

.70 







q 18 

.49 

.41 

.40 

.40 

.74 

.71 

.80 






q28 

.41 

.43 

.42 

.42 

.66 

.65 

.66 

.67 





q8 

.38 

.39 

.39 

.39 

.12 

.14 

.12 

.12 

.13 




ql4 

.35 

.37 

.36 

.36 

.11 

.13 

.11 

.11 

.12 

.56 



q20 

.40 

.43 

.42 

.42 

.13 

.15 

.13 

.13 

.14 

.66 

.61 


q24 

.36 

.37 

.36 

.36 

.11 

.13 

.11 

.11 

.12 

.56 

.52 

.61 
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Table 5. 

Reliability estimates of A TR scale 



Reliability 


Number of Items 

Factor 

Standardization 

Current 

Standardization 

Current 


sample 

sample 

sample 

sample 

Research usefulness 

0.92 

0.90 

9 

4 

Research anxiety 

0.92 

0.86 

8 

5 

Positive research 

0.93 

0.92 

8 

4 

predisposition 





Research difficulty 

0.71 

— 

3 

— 

Relevance to life 

0.77 

— 

4 

- 
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Figure 1. 

Factor structure of the Revised- Attitudes Toward Research Scale (R-ATR) 
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